Designing Nonlinear Turbo Codes 
with a Target Ones Density 

Jiadong Wang, Thomas Courtade, Tsung-Yi Chen, Bike Xie and Richard Wesel 
wjd@ee.ucla.edu, tacourta@ee.ucla.edu, tychen@ee.ucla.edu, xbk@ee.ucla.edu, wesel@ee.ucla.edu 



o 

oo 



CZ2 



> 

m 
in 

o 



X 



Abstract — Certain binary asymmetric cliannels, sucli as Z- 
channels in wliicli one of the two crossover probabilities is 
zero, demand optimal ones densities different from 50%. Some 
broadcast channels, such as broadcast binary symmetric channels 
(BBSC) where each component channel is a binary symmetric 
channel, also require a non-uniform input distribution due to 
the superposition coding scheme, which is known to achieve the 
boundary of capacity region. This paper presents a systematic 
technique for designing nonlinear turbo codes that are able to 
support ones densities different from 50%. To demonstrate the 
effectiveness of our design technique, we design and simulate 
nonlinear turbo codes for the Z-channel and the BBSC. The best 
nonlinear turbo code is less than 0.02 bits from capacity. 

I. Introduction 

Unlike their linear counterparts, nonlinear turbo codes are a 
family of turbo codes which can have average ones densities 
not equal to 50%. The constituent code symbols of nonlinear 
turbo codes are not restricted to linear combinations of state 
and input bits, and any ones density can be achieved by using 
a look-up table that maps the state and input bits to output 
bits. Parallel concatenated nonlinear turbo codes have been 
designed to maintain specific ones densities for multiple access 
channels H) and broadcast Z-channels |2|. 

In im, ill, the look-up tables defining nonlinear turbo codes 
were designed in an ad-hoc manner, mostly by hand. These 
techniques do not extend to trellises with many states. This 
paper provides a new and efficient technique for designing 
nonlinear turbo codes. The advantage of this technique is 
that the complexity does not grow exponentially with the 
number of states. We demonstrate the effectiveness of the 
codes designed using our technique on both the point-to- 
point Z-channel and the broadcast binary symmetric channel 
(BBSC). 

This paper is organized as follows. Section |II] introduces 
nonlinear turbo codes and describes an efficient design scheme 
in a step-by-step manner Section |lll] describes nonlinear 
turbo codes designed specifically for the Z-channel and the 
BBSC and provides simulation results. Section HVl delivers the 
conclusions. 

II. A Design Scheme for Nonlinear Turbo Codes 

A nonlinear turbo code is defined by look-up tables that 
map state and input bits to output bits in the trellises of 
the constituent codes. By controlling the number of ones in 
the look-up table, any desired ones density can be closely 
approximated. In general, a brute-force search to find the 
look-up table yielding the largest effective free distance is 



impractical because the complexity grows exponentially with 
the number of states. 

In this section, we propose a systematic method for design- 
ing nonlinear turbo codes with a complexity that does not grow 
exponentially in the number of states. Therefore, our approach 
can be used to design codes employing trellises with many 
states. In Section HUl we use this technique to design several 
capacity-approaching nonlinear turbo codes for the Z-channel 
and the BBSC. We would like to remark that although our 
technique is described in the context of binary turbo codes 
that use identical constituent encoders, it immediately extends 
to more general cases. 

A. State sub-tables, branch distance, and merge distance 

Throughout this section, we consider a trelhs corresponding 
to a constituent code of the turbo code. We assume that the 
trellis has t states and each trellis transition corresponds to k 
input bits and n output bits. Using this notation, we define a 
state sub-table. 

Definition 1: A state sub-table M{s) corresponding to 
state s is a 2'''-by-n binary matrix describing the mapping 
of input-bits to output-bits when the encoder is in state s 
as follows. When the encoder is in state s and the input- 
bits are bi,...,bk, the encoder outputs the n bits in the 
row corresponding to the fc-tuple bi, . . . ,bk- Without loss of 
generality, we assume that the rows of M{s) are indexed by 
the binary fc-tuples in lexicographical order 

We also define the minimum branch-distance and the min- 
imum merge-distance as follows: 

Definition 2: The branch-distance of state s is the mini- 
mum distance between the rows of M{s), the branch-distance 
of a trellis is the minimum of the state branch-distances. The 
merge-distance of a state and the merge-distance of a trellis 
are defined analogously as the minimum distances between 
the n-bit outputs of trellis transitions that merge into a state. 

Note that the particular distance metric can depend on the 
channel, as we will see in Section Hill 

As discussed above, designing the constituent codes for a 
nonlinear turbo code includes defining a look-up table that 
maps state and input bits to output bits in a trellis. A look- 
up table that defines a constituent code with a large effective 
free distance is preferable to one that defines a constituent 
code with a small effective free distance. A good heuristic for 
determining whether a look-up table will produce a code with 
a large effective free distance is to analyze the distances at the 



branches and merges of the trelHs. Our approach is based on 
the following key observation: 

Observation 1: If Hi and 112 are 2^ x 2^ and n x n per- 
mutation matrices respectively, then a state sub-table M{s') = 
IIiAf (s)n2 has the same branch distance properties as M{s) 
in the sense that the set of distances between outputs from 
state s is the same as the set of distances between outputs 
from state s'. 

B. Description of Design Scheme 

With this observation in mind, we now describe our design 
approach: 

1) If we require a constituent code with ones density ui, 
choose V to be the nearest integer to ui • n • 2'^. 

2) Select parameters df, and d„i, where df, is the desired 
branch-distance of the trellis and dm is the desired 
merge-distance of the trellis. 

3) Create a state sub-table A/(l) with exactly v ones. This 
ensures that the average ones density of the output bits 
from state 1 is -^p^ which is approximately ui. The 
state sub-table is designed by carefully placing the ones 
so that the branch-distance is greater than d\j. If this is 
not possible, then return to step 2 and select a smaller 
dft. This step is the main source of complexity. If n and 
k are sufficiently small, a good A/(l) can be found via 
a brute-force search. 

4) For each other state s E {2,...,i}, choose random 
permutation matrices Hi and 112 and set M{s) = 
niM(l)n2. By Observation 1, this ensures that the 
trellis has branch-distance greater than db- 

5) Check the resulting merge-distance of the trellis. If it is 
less than dm, return to step 4. If a maximum number of 
iterations is reached, return to step 2 and select a smaller 

6) Check the effective free distance of the obtained code. 
Steps 3-5 are generally repeated several times to produce 
several candidate codes. Usually, we select the code with 
the largest effective free distance. 

With this scheme, we can construct nonlinear turbo codes 
without an exhaustive computer search over all lookup tables 
with the desired ones density. Since we only manually design 
a sub-table for one state, the complexity does not grow 
exponentially in the number of states. Through a series of 
many experiments, we have observed that this procedure is 
effective for designing nonlinear turbo codes that approach 
capacity in channels demanding nonuniform ones densities. 
In Section [nil we use this technique to design nonlinear turbo 
codes for Z-channels and BBSCs. 

III. Examples 

In this section, we apply our design technique to two differ- 
ent types of channels: Z-channels and BBSCs. In each case, 
we successfully design nonlinear turbo codes that approach 
the capacity of their respective channels. For the Z-channel, 
the codes are designed to perform well with respect to the 



directional Hamming distance metric, while in the case of the 
BBSC, the traditional Hamming distance metric is used. 

A. Z-channels: Introduction and directional distance 

Z-channels are binary asymmetric channels in which one 
of the two crossover probabilities is zero (see Figure [T]|. This 
subsection considers point-to-point Z-channels and uses the 
model in which the probability of the 1 — !> crossover is 0. 
This channel applies to certain data storage systems |3| and 
certain optical communication systems f?). 

Golomb |5| studied the capacity and optimal ones density 
of Z-channels. The capacity of the Z-channel is given by 



C^H{uo{l-p))-uoH{p) 



with optimal zeros density 



Uq 



P 



,P/(1-P) 



(1) 



(2) 



1 + (1 -p)pP/(i-p)' 

The optimal ones density for the Z-channel is higher than 
50% everywhere except for the noiseless channel where the 
crossover probability is zero. All linear codes have an average 
ones density of 50%, and are thus prevented from achieving 
the capacity of the Z-channel. However, nonlinear turbo codes 
can provide any ones density and thus have the potential to 
achieve the capacity of the Z-channel. 

For the Z-channel, directional Hamming distance, intro- 
duced in 1 3 1, accurately describes the distance between code- 
words under Z-channel distortion. Consider two codewords of 
length-n bits, X and X. The directional Hamming distance 
between X and X is 



dD{X,X)^ J2 I{x^=0,i^^l) 



(3) 



where / is the indicator function. The directional Hamming 
distance is asymmetric, and di:){X,X) is usually not equal to 
doiX, X). A more precise definition of the pair-wise distance 
of Z-channel is defined in [Tl and ||6l: 



dziX,X)=max dDiX,X),dD{X,X) 



(4) 



since the larger directional distance matters in the decoding. 

B. Z-channels: Code Design 

The design begins with the 16-state duo-binary trellis 
described in |2|. With the design technique described in 
Subsection III-BI we first design a systematic rate-1/10 turbo 
code with a target ones density of 0.621. The actual ones 
density produced is 0.5953. This rate-1/10 code maximizes 
the minimum pairwise directional Hamming distance for the 
splits from each state and the merges to each state. 

We choose the best code with largest directional effective 
free distance among the randomly generated nonlinear turbo 
codes. Table U shows the nine nonsystematic (parity) bits for 
the two identical constituent nonlinear binary trellis encoders 
that comprise this turbo code. The two constituent encoders 
each produce nine parity bits, which join with the two sys- 
tematic bits to produce 20 coded bits for each two-bit input 



TABLE I 

Octal labeling for constituent binary trellis codes . Rows 

REPRESENT THE STATE S1S2S3S4, COLUMNS REPRESENT THE INPUTMiM2- 



State 


input 1 


00 


01 


10 


11 


0000 


534 


343 


671 


517 


0001 


476 


073 


707 


364 


0010 


346 


257 


571 


632 


0011 


137 


752 


711 


265 


0100 


754 


566 


227 


171 


0101 


370 


467 


516 


335 


0110 


743 


574 


037 


626 


0111 


566 


273 


532 


615 


1000 


465 


457 


343 


334 


1001 


752 


665 


037 


370 


1010 


274 


563 


754 


307 


1011 


723 


354 


617 


465 


1100 


435 


643 


317 


564 


1101 


153 


666 


703 


334 


1110 


327 


176 


453 


664 


nil 


466 


153 


335 


761 



TABLE II 
Octal puncturing patterns (in octal) of parity bits . Punctured 

BITS are indicated BY 1' S . THE PUNCTURING PERIOD IS 9 BITS . 



Rate 


Encoder 1 


Encoder 2 


Ones Density 


Optimal Density 


1/10 


000 


000 


0.5953 


0.621 


1/9 


001 


002 


0.5955 


0.6197 


1/8 


201 


042 


0.5938 


0.6181 


1/7 


241 


043 


0.5915 


0.6161 


1/6 


243 


243 


0.5911 


0.6134 


1/5 


247 


263 


0.5828 


0.6094 


1/4 


257 


267 


0.5742 


0.6035 


1/3 


277 


367 


0.5599 


0.5931 



symbol. The interleaver for this turbo code can be found in 

Q. 

As is commonly done with turbo codes, we puncture this 
rate-1/10 code to create a variety of rates. The puncturing 
patterns are given in Table |ll] with the code rates and ones 
densities they produce. In each case, the resulting ones density 
closely approximates the optimal ones density. 

C. Z-channels: Numerical Results 

We simulated our nonlinear turbo codes on Z-channels with 
capacities slightly greater than the code rates. All of the sim- 
ulations use 20,000 input bits per codeword and an extended 
spread interleaver [8]. The complete interleaver description is 
available online jT). Figure |2] shows the capacity of the Z- 
channel for different crossover probabilities and the observed 
operating point (code rate and crossover probability) where 
each codes achieved bit error rates of less than 10^^. The 
distance from capacity ranges from approximately 0.018 to 
0.05 bits. 

D. BBSC: Introduction 

We also designed nonlinear codes for the two-user BBSC, 
which consists of two binary symmetric component channels, 
one with transition probability a and the other with transition 
probability j3, as shown in Figure[3] Without loss of generality, 
we assume a < (3. A simple and optimal encoding scheme 
is an independent-encoding approach in which symbols from 
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Fig. 1. Channel model for the Z-channel. 
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Fig. 2. The capacity with optimal ones density and designed nonlinear Turbo 
codes for Z-channels. The operating points from left to right correspond to 
the rate-1/3 to rate-1/10 nonUnear turbo codes respectively. 



independent codebooks are added together using the XOR 
function. We refer to this scheme as superposition coding, and 
the encoder structure is shown in Figure |4] 

The capacity region of a degraded broadcast channel was 
established by Cover [9], Bergmans ifTOl and Gallager ifTTIl . 
Cover 1 12 1 introduced an independent-encoding scheme for 
two-user broadcast channels. This scheme is known to achieve 
the boundary of the capacity region for the broadcast binary- 
symmetric channel (BBSC) and is investigated in jfTSJJ lfT4l 
L15J Li6l. 

The capacity region for a BBSC is given by 



Ri < h{c 



{a * pi) — h{a) 
R2<l~h{/3*pi), 



(5) 



where pi is the ones density of Xi, 
and the operation * is defined by 



X2 has 50% ones density 



a*b = a{l -b) + bil- a), < a,b < I. 



(6) 



In order to use a superposition coding scheme, the codes 
of the two users cannot both have ones densities of 50%. 
This precludes the exclusive use of linear codes. Using the 
techniques described in Section III-BI we design a family 
of nonlinear turbo codes that can provide a controlled ones 
density. Superposition of one of our nonlinear codes with a 
linear turbo code produces an overall transmission with the 
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Fig. 5. 64-state 4-input turbo encoder trellis structure 
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Fig. 3. Channel model for two-user broadcast binary-symmetric channel 
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Fig. 4. Encoder scheme for two-user broadcast binary-symmetric channel ^-^ g Achievable capacity region and simulated points for superposition 

codes and time-sharing schemes for BBSC 



potential to approach an optimal point the capacity region of 
the BBSC. 

E. BBSC: Code Design 

This section describes the construction of practical super- 
position codes for the two-user BBSC under different channel 
scenarios. For the second user, we use a linear turbo code 
from the DVB-RCT standard, which has a 16-state duo- 
binary turbo encoder trellis structure, and extend it to lower 
rates. For the first user we use nonlinear turbo codes with 
various ones densities, a 16-state duo-binary turbo encoder 
trellis structure from lil7J when the channel parameters of 
Fig. E] are (a,/3) = (0.188,0.2017), and the 64-state 4- 
input turbo encoder trellis structure shown in Figure |5] when 
(a,/3) = (0.01,0.108). The design process is described as 
follows: 

1) Choose an appropriate ones density for the first user 
As shown in (|5]l, for a BBSC with fixed a and j3, the 
boundary of the capacity region is a function of pi, the 
ones density of User 1 : 

Rl=fl{Pl),R2^f2{Pl)- 

where /i and /2 are monotonically increasing and 
decreasing functions of pi respectively. Since these 
functions are one-to-one for < pi < 0.5, pi can be 



calculated by 

Pl=f^\Rl) = f2\R2)- 

Thus under this channel, for any working point (ri, r2), 
where ri and r2 are the rates of user 1 and user 2, pick 
the ones density pi that satisfies 

f2\r2)<pi<f[\n) 

where fi^{ri) ( /2~^('^2) ) corresponds to the ones den- 
sity of the intersection between the horizontal (vertical) 
line from (ri,r2) and the capacity curve. 

2) For the desired pi, use the approach in Section Ill-Bl to 
design the look-up table for the nonlinear turbo code. 
The look-up tables, interleavers and puncture patterns 
could be found online 1 7 1 . 

3) If a low ones density and a high rate is needed, there will 
be multiple all zero outputs in the look-up table, which 
is problematic. Thus we design a lower rate codes with 
the same ones density by using step 2 above. Then we 
uniformly puncture the codewords to obtain the desired 
rate while preserving the desired rate. 

F. BBSC: Numerical Results 

We simulated the superposition codes employing our non- 
linear turbo codes with different rates for a variety of BBSCs. 
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Fig. 7. A special case a = 0, /3 = 0.1: capacity region and simulated points 
for superposition codes and time-sharing schemes for the BBSC 



Figure |6] shows the capacity region for 2 different channel sce- 
narios and four operating points for each case (each operating 
point has a BER less than < 10"^). 

All of the simulations use 20,000 input bits and an ex- 
tended spread interleaver. For the nonlinear codes, we use 
an iV = 10000 symbol- wise interleaver in the case where 
(a,/3) = (0.188,0.2017), and we use an TV = 5000 symbol- 
wise interleaver in the case where {a, (3) = (0.01,0.108). 
For the linear code, we use an iV = 10000 symbol-wise 
interleaver. 

One special case of the BBSC is when (a,/?) = (0,0.1). 
This coiTesponds to the case when the first channel is noiseless 
and the distortion observed by User 1 is solely due to interfer- 
ence from User 2's superposed codeword. The capacity region 
in this scenario is 



Ri < h{pi) 

R2 < l-h{l3*pi). 



(7) 



Since the first channel is noiseless, we can use permutation 
codes ifTSJI whose rate can achieve h{pi) for given ones density 
pi, and traditional turbo codes for the second user. Simulation 
results are given in Figure |7] We would like to point out that 
some of the operating points perform at rates higher than those 
theoretically achievable by time-sharing schemes. 

G. Remarks 

Linear turbo codes with 50% ones density can also work un- 
der asymmetric channels with a loss in performance. In some 
channels, this loss could be small due to the small gap between 
the capacity with optimal ones density and mutual information 
with 50% onesd density. However, for certain channels such as 
the multiple access OR channel, the theoretical gap is large and 
this new design technique of efficiently constructing nonlinear 
turbo codes is useful especially for a large trellis. 



IV. Conclusions 

This paper proposes a systematic method for designing 
nonlinear turbo codes with a desired ones density. Using our 
approach, we designed a series of nonlinear turbo codes that 
approached capacity in several Z-channels. These codes were 
designed using the directional Hamming distance metric. We 
also designed a series of nonlinear turbo codes which were 
scombined via superposition with linear turbo codes for use 
on the two-user BBSC. These codes were designed using 
the traditional Hamming distance metric and also performed 
near capacity. The presented design techniques for nonlinear 
turbo codes could also be extended to general group-addition 
degraded broadcast channels |19| as future work. 
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